RESUMO
OBJECTIVE: Length of stay (LOS) is an important metric for the organization and scheduling of care activities. This study sought to propose a LOS prediction method based on deep learning using widely available administrative data from acute and emergency care and compare it with other methods. PATIENTS AND METHODS: All admissions between January 1, 2011 and December 31, 2019, at 6 university hospitals of the Hospices Civils de Lyon metropolis were included, leading to a cohort of 1,140,100 stays of 515,199 patients. Data included demographics, primary and associated diagnoses, medical procedures, the medical unit, the admission type, socio-economic factors, and temporal information. A model based on embeddings and a Feed-Forward Neural Network (FFNN) was developed to provide fine-grained LOS predictions per hospitalization step. Performances were compared with random forest and logistic regression, with the accuracy, Cohen kappa, and a Bland-Altman plot, through a 5-fold cross-validation. RESULTS: The FFNN achieved an accuracy of 0.944 (CI: 0.937, 0.950) and a kappa of 0.943 (CI: 0.935, 0.950). For the same metrics, random forest yielded 0.574 (CI: 0.573, 0.575) and 0.602 (CI: 0.601, 0.603), respectively, and 0.352 (CI: 0.346, 0.358) and 0.414 (CI: 0.408, 0.422) for the logistic regression. The FFNN had a limit of agreement ranging from -2.73 to 2.67, which was better than random forest (-6.72 to 6.83) or logistic regression (-7.60 to 9.20). CONCLUSION: The FFNN was better at predicting LOS than random forest or logistic regression. Implementing the FFNN model for routine acute care could be useful for improving the quality of patients' care.
Assuntos
Serviços Médicos de Emergência , Hospitalização , Humanos , Tempo de Internação , Hospitais , Redes Neurais de Computação , Estudos RetrospectivosRESUMO
Background: Unwarranted extended length of stay (LOS) increases the risk of hospital-acquired complications, morbidity, and all-cause mortality and needs to be recognized and addressed proactively. Objective: This systematic review aimed to identify validated prediction variables and methods used in tools that predict the risk of prolonged LOS in all hospital admissions and specifically General Medicine (GenMed) admissions. Method: LOS prediction tools published since 2010 were identified in five major research databases. The main outcomes were model performance metrics, prediction variables, and level of validation. Meta-analysis was completed for validated models. The risk of bias was assessed using the PROBAST checklist. Results: Overall, 25 all admission studies and 14 GenMed studies were identified. Statistical and machine learning methods were used almost equally in both groups. Calibration metrics were reported infrequently, with only 2 of 39 studies performing external validation. Meta-analysis of all admissions validation studies revealed a 95% prediction interval for theta of 0.596 to 0.798 for the area under the curve. Important predictor categories were co-morbidity diagnoses and illness severity risk scores, demographics, and admission characteristics. Overall study quality was deemed low due to poor data processing and analysis reporting. Conclusion: To the best of our knowledge, this is the first systematic review assessing the quality of risk prediction models for hospital LOS in GenMed and all admissions groups. Notably, both machine learning and statistical modeling demonstrated good predictive performance, but models were infrequently externally validated and had poor overall study quality. Moving forward, a focus on quality methods by the adoption of existing guidelines and external validation is needed before clinical application. Systematic review registration: https://www.crd.york.ac.uk/PROSPERO/, identifier: CRD42021272198.
RESUMO
Objective: Systematic review of length of stay (LOS) prediction models to assess the study methods (including prediction variables), study quality, and performance of predictive models (using area under receiver operating curve (AUROC)) for general surgery populations and total knee arthroplasty (TKA). Method: LOS prediction models published since 2010 were identified in five major research databases. The main outcomes were model performance metrics including AUROC, prediction variables, and level of validation. Risk of bias was assessed using the PROBAST checklist. Results: Five general surgery studies (15 models) and 10 TKA studies (24 models) were identified. All general surgery and 20 TKA models used statistical approaches; 4 TKA models used machine learning approaches. Risk scores, diagnosis, and procedure types were predominant predictors used. Risk of bias was ranked as moderate in 3/15 and high in 12/15 studies. Discrimination measures were reported in 14/15 and calibration measures in 3/15 studies, with only 4/39 externally validated models (3 general surgery and 1 TKA). Meta-analysis of externally validated models (3 general surgery) suggested the AUROC 95% prediction interval is excellent and ranges between 0.803 and 0.970. Conclusion: This is the first systematic review assessing quality of risk prediction models for prolonged LOS in general surgery and TKA groups. We showed that these risk prediction models were infrequently externally validated with poor study quality, typically related to poor reporting. Both machine learning and statistical modelling methods, plus the meta-analysis, showed acceptable to good predictive performance, which are encouraging. Moving forward, a focus on quality methods and external validation is needed before clinical application.
RESUMO
OBJECTIVE: This systematic review sought to establish a picture of length of stay (LOS) prediction methods based on available hospital data and study protocols designed to measure their performance. MATERIALS AND METHODS: An English literature search was done relative to hospital LOS prediction from 1972 to September 2019 according to the PRISMA guidelines. Articles were retrieved from PubMed, ScienceDirect, and arXiv databases. Information were extracted from the included papers according to a standardized assessment of population setting and study sample, data sources and input variables, LOS prediction methods, validation study design, and performance evaluation metrics. RESULTS: Among 74 selected articles, 98.6% (73/74) used patients' data to predict LOS; 27.0% (20/74) used temporal data; and 21.6% (16/74) used the data about hospitals. Overall, regressions were the most popular prediction methods (64.9%, 48/74), followed by machine learning (20.3%, 15/74) and deep learning (17.6%, 13/74). Regarding validation design, 35.1% (26/74) did not use a test set, whereas 47.3% (35/74) used a separate test set, and 17.6% (13/74) used cross-validation. The most used performance metrics were R2 (47.3%, 35/74), mean squared (or absolute) error (24.4%, 18/74), and the accuracy (14.9%, 11/74). Over the last decade, machine learning and deep learning methods became more popular (P=0.016), and test sets and cross-validation got more and more used (P=0.014). CONCLUSIONS: Methods to predict LOS are more and more elaborate and the assessment of their validity is increasingly rigorous. Reducing heterogeneity in how these methods are used and reported is key to transparency on their performance.
